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Abstract — In this paper we design non-uniform bit-wise 
puncturing distributions for irregular non-binary LDPC (NB- 
LDPC) codes. The puncturing distributions are optimized by 
minimizing the decoding threshold of the punctured LDPC code, 
the threshold being computed with a Monte-Carlo implementa- 
tion of Density Evolution. First, we show that Density Evolution 
computed with Monte-Carlo simulations provides accurate (very 
close) and precise (small variance) estimates of NB-LDPC code 
ensemble thresholds. Based on the proposed method, we analyze 
several puncturing distributions for regular and semi-regular 
codes, obtained either by clustering punctured bits, or spreading 
them over the symbol-nodes of the Tanner graph. Finally, 
optimized puncturing distributions for non-binary LDPC codes 
with small maximum degree are presented, which exhibit a gap 
between 0.2 and 0.5 dB to the channel capacity, for punctured 
rates varying from 0.5 to 0.9. 

I. Introduction 

In the modern approach of error correcting codes, binary 
Low-Density Parity-Check (LDPC) codes are playing a very 
important role due to their low decoding complexity and 
to the fact that they can be optimized for a broad class 
of channels, with asymptotic performance approaching the 
theoretical Shannon limit [JJ. On the other hand, LDPC codes 
over non-binary alphabets exhibit better performance for short 
to medium code lengths f2], without sacrifying the almost 
optimal asymptotic performance. The performance gain at 
short length comes at an additionnal cost in the decoding 
complexity. However, this additionnal cost can be reduced 
by using sub-optimal low complexity algorithms, as proposed 
e.g. in 131, Q. When coded bits are transmitted over time- 
varying channels, as for wireless communication systems, the 
rate of the forward error correction code has to be adapted 
to the quality of the channel, so as to achieve both high 
throughput and reliability. Rate adaptation requires codes of 
different rates, which can be efficiently obtained by using one 
low rate mother code and puncture it to get higher rates. The 
advantage of puncturing is that the same decoder can be used 
regardless the puncturing pattern: according to the channel 
conditions, the transmission system adapts by just changing 
the puncturing pattern at the transmitter, and the depuncturing 
pattern at the receiver 

In this work we explore the rate-adaptability behavior of 
non-binary LDPC codes. In the binary case. Ha et al. |5| 
showed that puncturing patterns can be optimized so that 
rate-compatible puncturing results in a small loss of the 
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asymptotical performance (threshold). For non-binary codes, 
Kline et al. |6| proposed a design of puncturing patterns based 
on symbol recovery levels, similar to the design proposed 
in the binary case in |7|. Given a target punctured rate, the 
algorithm proposed in [TJ computes a puncturing pattern that 
minimizes the number of recovery steps for the punctured bits. 
This approach is thought for finite length puncturing design, 
but does not give insights regarding the optimization of the 
puncturing distribution, for LDPC code ensembles. 

In the binary case, the puncturing distribution is defined by 
fractions {fd}d of punctured bit-nodes of degree d. For non- 
binary codes, the puncturing distribution can be defined in 
terms of fractions {fd,k}d,k of degree-d symbol-nodes with 
exactly k punctured bits per symbol. Note that addressing 
the problem of the optimization of puncturing distributions 
is compliant with the finite length approach of f6l, and from 
an optimized puncturing distribution, one can advantageously 
use the algorithm proposed in |6| in order to construct the 
actual puncturing pattern Q. 

Although the study of non-binary LDPC codes is mainly 
motivated by their performance at short to medium lengths, 
their asymptotic analysis proves also to be very useful. In 
particular, it allows the understanding of topology indepen- 
dent phenomena or, stated differently, it allows distinguishing 
between events due to code family parameters, and those due 
to a particular code realization. The asymptotical performance 
of an ensemble of codes is referred to as threshold, and corre- 
sponds to the worst channel condition that allows transmission 
with an arbitrary small error probability when the code length 
tends to infinity. It can be determined by the concept of density 
evolution 18]. Unfortunately, for the non-binary case, exact 
density evolution is only manageable for the Erasure Channel 
m, ifTol . In the case of more general memryless channels, 
like the binary-input AWGN channel, density evolution must 
be approximated using numerical techniques, or approched by 
modeling the density of the messages, e.g. with a Gaussian 
approximation ifTTI . 

In this work we show that density evolution of non-binary 
LDPC codes can be tightly approximated by using fast Monte- 
Carlo approximation of the density of the messages exchanged 

'As a parallel, consider the construction of bipartite Tanner graphs: for 
given node-degree distributions, we can advantageously use the Progressive 
Edge Growth (PEG) algorithm, so as to maximize the girth of the constructed 
graph; however, if the PEG algorithm is not constrained with a given degree 
distribution, the algorithm will tend to construct a regular graph, which does 
not provide the best performance 



during the iterative decoding of an infinite code. This method 
allows for obtaining very close estimates of thresholds of 
non-binary ensembles, making possible the optimization of 
non-binary codes for a wide range of applications and chan- 
nel models. In particular, it can be successfully applied for 
optimizing puncturing distributions for rate-compatible non- 
binary LDPC codes. We used genetic optimization methods for 
searching good puncturing distributions for non-binary LDPC 
codes over Fig, with maximum symbol-node degree equal to 
10. Optimized distributions exhibit a gap to capacity between 
0.2 and 0.5dB, for punctured rates varying from 0.5 to 0.9. 

The rest of the paper is organized as follows. In Section 
nil we introduce the notations and review some basics on non 
binary LDPC codes. Section Hill presents the proposed method 
for approximating the density evolution of non-binary LDPC 
codes. Puncturing distributions are introduced in Section HV] 
and clustering vs. spreading distributions are analyzed for 
regular and semi -regular codes in Section [Vj The optimization 
of puncturing distributions for irregular codes is addressed in 
Section |Vl| Finally, Section IVlIl concludes the paper. 

II. NON BINARY LDPC CODES 

We consider non-binary codes defined over ¥q, the finite 
field with q elements, where g = 2^ is a power of 2 (the last 
condition is only assumed for practical reasons). We fix once 
for all a vector space isomorphism: 

F, ^ ¥1 (1) 

and we say that {xq, . . . , Xp^i) G F2 is the binary image of the 
symbol X E¥q if they correspond to each other by the above 
isomorphism. An LDPC code over ¥q is defined as the kernel 
of a sparse parity-check matrix H G M.M.N(^q)- The matrix 
H can be represented by a bipartite graph comprising M 
constraint-nodes and N symbol-nodes, associated respectively 
with its M rows and its N columns. A constraint-node and a 
symbol-node are connected by an edge if the corresponding 
entry of H is non-zero. Each edge of the graph is further 
labeled by the corresponding non-zero entry. Thus, symbol- 
nodes represent coded symbols, while constraint-nodes to- 
gether with edge labels correspond to linear constraints on 
these symbols. The number of edges incident to a node is 
referred to as node degree. Interconnections between nodes are 
usually expressed by the edge-perspective degree distribution 
polynomials X{x) = Y.d'^dx'^^^ and p{x) = Y.dPdx'^^^, 
where Xd and pd represent represent the fraction of edges 
connected respectively to symbol and constraint-nodes of 
degree d. We denote by r the design coding rate, which 

is given by r = 1 — tt^-t-tj-- It is well known that 

Jo 

the asymptotic performance of binary LDPC codes depends 
only on these degree distribution polynomials [8]. For non- 
binary codes, the performance can be slightly improved by 
choosing optimized distributions of edge labels [10]. However, 
in general this improvement seems to be minimal; therefore, 
in the next section we will consider ensembles £q{X,p), 
of non-binary LDPC codes over ¥q, with edge-perspective 



degree distribution polynomials A and p, and with edge labels 
uniformly distributed over F*. 

III. Asymptotic performance of non-binary LDPC 

CODES 

Consider some channel model depending on a parameter 
s, such that the channel conditions worsen when s increases 
(for instance, the noise variance for the AWGN channel, or 
the erasure probability for the BEC channel). The threshold 
of the ensemble £q{X, p) is defined as the supremum value of 
s (worst channel condition) that allows transmission with an 
arbitrary small error probability, assuming that the transmitted 
data is encoded with an arbitrary-length code of £q{X, p). For 
symmetric channels, we can assume that the all-zero codeword 
is transmitted, and the threshold can be approximated by 
simulating the density evolution of messages exchanged within 
the iterative decoding of an "infinite" code from the ensemble 
1 12|. For doing so, we have to ensure that exchanged messages 
remain uncorrelated for a large number of iterations. Since the 
density evolution is observed on a finite number of exchanged 
messages, we proceed as described below in order to decor- 
relate them. Consider that we have to compute an outgoing 
message (either from a symbol, or a constraint-node). First, 
we sample the node degree d according to the corresponding 
degree distribution (either A, or p), and we randomly choose 
d — 1 messages from those produced at the previous half 
iteration. These messages are considered as being received 
by the node on the d — 1 incoming edges, and d — 1 edge 
labels are also uniformly sampled from F*. Besides, in order 
to compute outgoing messages from symbol-nodes, we also 
need the knowledge the a priori information corresponding to 
the channel output. In this case, for each new outgoing symbol- 
node message, the channel output is also resampled according 
to the channel model. The outgoing message is then computed 
according to the Sum-Product decoding rules llT3l . This is 
illustrated at Figure [T] for X{x) = ^x + jX^ and p{x) = x^ 
(designed code rate r = 1/2), where outgoing symbol and 
constraint-node messages are respectively denoted by ai and 
13 j, edge labels are denoted by hi j, and the a priori information 
corresponding to the channel output by 7^. We also remark that 
exchanged messages are better decorrelated for higher order 
finite fields, due to the edge-labels resampling. 




half iteration half iteration 



Fig. I. Density evolution approximation by Monte-Carlo simulation 



TABLE I 

Thresholds for binary irregular LDPC codes 
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0.808 


0.904 


0.876 


0.916 


0.757 


1.2 
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0.919 


0.730 


0.911 


0.806 


0.923 


0.691 


1.1 
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0.930 


0.627 


0.907 


0.774 


0.931 


0.622 


1.0 


7 


0.942 


0.515 


0.922 


0.699 


0.935 


0.577 


1.7 
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0.950 


0.448 


0.938 


0.557 


0.947 


0.468 


1.5 
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0.954 


0.409 


0.940 


0.533 


0.954 


0.411 


1.1 


10 


0.956 


0.393 


0.942 


0.523 


0.955 


0.403 


1.2 


11 


0.957 


0.380 


0.942 


0.518 


0.957 


0.377 


1.1 


12 


0.958 


0.373 


0.942 


0.513 


0.958 


0.367 


0.8 


15 


0.962 


0.335 


0.944 


0.503 


0.962 


0.340 


1.6 


20 


0.965 


0.310 


0.946 


0.482 


0.963 


0.325 


0.7 


30 


0.969 


0.273 


0.946 


0.469 


0.965 


0.299 


1.6 


50 


0.972 


0.248 


0.952 


0.428 


0.971 


0.260 


0.8 


RMSE 


0.0168 


0.1459 


0.0031 


0.0272 





Table U presents a comparison of thresholds obtained by 
the proposed method (MC-DE) and those obtained by exact 
density evolution, for irregular binarjH LDPC codes with 
rate 1/2 over the AWGN channel. Thresholds are are shown 
both in terms of noise variance and corresponding Eb/No 
value. The maximum bit-node degree is reported in the left 
column; the exact degree distributions A and p can be found 
in Tables I and II. For comparison purposes, we have also 
included threshold values computed by using the Gaussian- 
Approximation proposed in 114|. At each half-iteration of the 
Monte-Cai-lo Density-Evolution (MC-DE) simulation, 10000 
outgoing messages are computed, and a maximum number 
of 500 iterations are executed. For each pair of degree 
distributions (A,p), several threshold estimations have been 
performed; the mean value is reported in the MC-DE/ci* 
column, and the standard deviation (xlO"^) is reported in the 
last column. As it can be observed, the standard deviation 
is between 0.0008 and 0.0017; hence we conclude that the 
proposed method is precise. Furthermore, the Root Mean 
Square Error (RMSE) of estimated thresholds (reported on the 
last row) if equal to 0.0031, which proves that the proposed 
method is also accurate. 

IV. Puncturing distributions for non-binary 

LDPC CODES 

Let a non binary LDPC code be used over a binary-input 
channel: a non-binary codeword is mapped into its binary 
images by using ([TJ, then transmitted over the channel. Hence, 
even if non-binary codes operate at the symbol level, non- 
binary codewords still can be punctured at the bit-level. A 
coded symbol can be either completely or partially punctured, 
according to whether all the bits of its binary image are 
punctured or not. In order to design good puncturing patterns 
we have to answer the following questions: 
• Assuming that the code is regular, should puncturing bits 
be clustered or spread over the symbol-nodes? Clustering 
yields a reduced number of completely punctured symbols, 
which receive no information from the channel. On the 

-Exact DE for non-binary LDPC codes over the AWGN channel is not 
known. 




Fig. 2. Bit level puncturing pattern for non-binary LDPC codes 



contrary, spreading results into an increased number of 
partially punctured symbols, which still benefit from some 
partial information from the channel. 

• In case of irregular codes, how should punctured bits be 
chosen with respect to symbol-node degrees? 

Let fd.k denote the fraction of degree-d symbol-nodes with k 
punctured bits. Note that X]a;=o fd-f^ ^ symbol-node 
contains either fc = (unpunctured), or k — 1, . . . , or k = p 
punctured bits. The overall fraction of punctured bits, /, and 
the corresponding punctured rate, r^, are given by: 

^ d=l k=Q ■' 

where dg is the maximum symbol-node degree, and Ld ~ 

J — is the fraction of symbol-nodes of degree d. 

d A(a;) da; 

This is illustrated at Figure |2] for an LDPC code over Fg with 
designed code rate r = 1/2, where /2,i = /2,2 — 1/4, and 
/4,3 = 1/4- Thus / = 0.25 and the designed punctured rate 
is r.p = 2/3. 

V. Analysis of puncturing distributions 

In this section we analyze different puncturing distributions 
for regular and semi-reg ulaJl non-binary LDPC codes over the 
BIAWGN channel. 

First of all, we consider three ensembles of regular LDPC 
codes with rate 1/2, and (symbol, constraint)-node degrees 
equal to (2,4), (3,6), or (4,8). A fraction / e [0,0.25] of 
coded bits are punctured, which corresponds to a punctured 
rate varying from Vp ~ 1/2 (no puncturing) to rp = 2/3. For 
each fraction of punctured bits, two puncturing distributions 
have been compared: 

• clustering distribution: punctured bits are clustered on a 
fraction / of completely punctured symbols-nodes; 

• spreading distribution: if / < | punctured bits are spread 
over a fraction pf of symbol-nodes, each one with one 
single punctured bit. Otherwise, all the symbol-nodes are 
punctured, each symbol-node containing either [pf\ or 
\pf~\ punctured bits 

Corresponding thresholds for codes over Fg and Fig are shown 
respectively at Figure |3] and Figure |4] We note the different 
behavior of these distributions, depending on the symbol-node 
degree: the spreading distribution provides better performance 
for the regular (2,4) code, but it is outperformed by the 
clustering distribution for the other two codes with higher 
symbol-node degrees. 




The intuition behind that goes as follows. An outgoing mes- 
sage from a degree-2 symbol-node depends on the information 
coming from the channel and the unique incoming extrinsic 
message. Thus, if a degree-2 symbol-node is completely 
punctured, it has no channel information, and the outgoing 
message is equal to the incoming one. This causes a lack 
of diversity in the iterative message-passing, which results in 
degraded decoding performance. 

Intermediary puncturing distributions can be obtained by 
mixing the above clustering and spreading distributions, as 
illustrated at Figure |5] for regular codes over F4: the fraction 
of punctured bits is / = 0.25, corresponding to a punctured 
rate = 2/3, and it is decomposed as / = /i + /2, where 
/i is the fraction of bits that are punctured in a spreading 
manner (in this case, 1 bit per symbol), and /2 is the fraction 
of bits that are punctured in a clustering manner (2 bits per 
symbol). Again, we can observe that the spreading distribution 
provides better performance for the regular (2, 4) code. Similar 
results are also shown at figure [8] for codes over Fig: for 
a clusterization degree k G {1,2,3,4} (on the abscissa), a 
fraction ^ of symbol-nodes are punctured, by punching k 
bits per symbol. Once more, we can observe that spreading 
is suitable for symbol-nodes of degree-2, while clustering is 
more appropriate for symbol-nodes of degree > 3. 

We investigate now different puncturing distributions for 
semi-regular LDPC codes, with node-perspective degree dis- 
tribution polynomials L{x) = ^x^ + ^x* and R{x) = x^, 

'with symbol-node degrees taking on two, relatively low, values. 



corresponding to a designed coding rate r ~ 1/2. The same 
fraction / = 0.25 of coded bits are punctured, so as to obtain 
a punctured rate rp — 2/3. The fraction is decomposed as 
/ = /2 + /4, where /2 is the fraction of bits that are punctured 
on degree-2 symbol-nodes, and /4 is the fraction of bits that 
are punctured on degree-4 symbol-nodes. 

Numerical results are shown at Figure [T] and Figure |8] for 
solid curves, punctured bits are either clustered (full squares) 
or spread over the symbol-nodes (empty squares), for both 
symbol-nodes of degree 2 and 4. For the dashed curve, 
punctured bits are spread over symbol-nodes of degree 2, and 
clustered over symbol-nodes of degree 4. In view of previous 
results for regular codes, we was expecting this last distribu- 
tion to provide the best performance. Surprisingly, it performs 
in between the clustering and the spreading distribution. What 
we observe is that in case there are punctured symbols of 
degree 4 (/4 > 0), the best performance is given by the 
clustering distribution. However, since all plots correspond to 
the same punctured rate, the best puncturing distribution in 
each figure is given by the lowest plot. In both cases this is 
the distribution that spreads all the punctured bits over symbol- 
nodes of degree 2 (/2 = 0.25, fi = 0, empty squarejj. This 
represents an important dissimilarity compared to the binary 
case, in which choosing all the punctured bits on degree-2 
bit-nodes proves to be catastrophic. In the non-binary case, 
spreading punctured bits over degree-2 symbol nodes yields 

"^However, the clustering distribution with /2 = 0.2 and /4 = 0.05 yields 
almost the same performance 
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TABLE II 

Optimized distribution, rr, 
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Fig. 7. Low vs. high symbol-degree, and spreading vs. clustering puncturing 
distributions for irregular LDPC codes over Fg 
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Fig. 8. Low vs. high symbol-degree, and spreading vs. clustering puncturing 
distributions for irregular LDPC codes over Fi6 



good performance, provided that the fraction of punctured bits 
is not too high. The next section will also provide evidence 
on this fact. 

VI. Optimization of puncturing distributions 

In this section we present optimized puncturing distributions 
for an irregular code over Fig with rate 1/2, and punctured 
code rates from 0.55 to 0.9. Optimization has been performed 
by using the Differential Evolution algorithm flSl. First of 
all we searched for good degree distribution pairs A and p 
of rate 1/2, with maximum symbol degreqfl equal to 10. 
The optimized distributions and the corresponding Et,/No 
threshold are given below: 



Xix) 
p{x) 



0.5376a; 0.1678x2 4 
0.5169a;4 4- 0.4831x5 
0.3638 



0.1360a;'' -h0.1586a;y 



Next, we searched for good puncturing distributions for punc- 
tured rates G {0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90}. 
Optimized distributions {fd.k] d = 2, 3, 5, 10 for punctured rates 
0.60, 0.75, and 0.90 are shown in Table |ll] Also indicated 
are kd, the average number of punctured bits per symbol-node 
of degree d, and fk, the average fraction of symbols with 
k punctured bits. As an equivalent representation, the corre- 
sponding clusterization distributions are shown at Figure |9l 

^Cf. discussion from the Introduction. 
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Fig. 9. Clusterization distributions for r'p g {0.55, . . . , 0.90} 



they consist of fractions of punctured symbol^ of degree-d, 
with k punctured bits per symbol, which are given by jzrf^^ 
for d = 2,3, 5, 10, and fc = 1, . . . , 4. These distributions seem 
to be random, and properties observed for regular codes shall 
not apply is this case. 

Thresholds of optimized punctured distributions are shown 
at Figure [TOl in terms of Eb/NQ. Are also plotted the theo- 
retical Shannon limit (capacity), and thresholds of distributions 
spreading punctured bits over degree-2 symbol-nodes. The gap 
between optimized distributions and capacity vary between 
0.18 db for — 0.5 (unpunctured code) and 0.52 dB 
for Tp — 0.9. We also note that the degree-2-spreading 
distribution yields very good performance up to punctured rate 
rp = 0.7. However, such a puncturing distribution proves to 
be catastrophic for punctured rates rp > 0.85 

Finally, Figure [TT| presents the Frame Error Rate perfor- 
mance of optimized distributions for finite code lengths. All 
the codes have binary dimension (number of source bits) 

'Recall that fd,k is the fraction of symbols of degree-d, with fc punctured 
bits per symbol 




equal to 4000 bits (1000 Fig-symbols). The mother code with 
rate 1/2 has been constructed by using the Progressive Edge 
Growth (PEG) algorithm |16|, and punctured patterns have 
been randomly chosen according to the optimized distribution. 
It is very likely that the performance can be further improved 
by using an optimized design of the puncturing patterns (in 
terms of symbol-recoverability steps, cf. discussion in the 
introduction), especially for high punctured rates. 

VII. Conclusions 

Except for simple channel models, as for instance the binary 
erasure channel, it is generally an arduous task to derive exact 
density evolution for non-binary LDPC codes. Yet, even if the 
exact density evolution were known, its inherent complexity 
would certainly prohibit its use for optimization purposes. 

In this paper we proposed a density evolution approximation 
method, by using Monte-Carlo simulation of an infinite code. 
The proposed method provides accurate and precise estimates 
of non-binary ensemble thresholds, and makes possible the op- 
timization of non-binary codes for a wide range of applications 
and channel models. 

As an application, rate-adaptability solutions for non-binary 
LDPC codes have been investigated. Puncturing distributions 
for regular and irregular codes have been analyzed by using 
simulated density evolution thresholds of non-binary LDPC 
codes over the AWGN channel. For regular codes, we showed 
that the design of puncturing patterns must respect different 
rules depending on the symbol-node degree: punctured bits 
must be spread over degree-2 symbol-nodes, while they must 
be clustered on symbol-nodes of higher degrees. If the number 
of punctured bits is relatively small, spreading punctured bits 
over degree-2 symbol-nodes yields also good performance for 
irregular codes. However, such a puncturing distribution could 
be catastrophic for higher punctured rates, in which case opti- 
mized puncturing distributions are indispensable. Finally, we 
presented optimized puncturing distributions for non-binary 
LDPC codes with small maximum degree, which exhibit a 
gap to capacity between 0.2 and 0.5 dB, for punctured rates 
varying from 0.5 to 0.9. 
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